# Multi-bit Quantization

INTELLECT-2-GGUF is the GGUF format quantized version of PrimeIntellect/INTELLECT-2, suitable for text generation tasks.

GGUF quantized version of the Gemma 3 27B instruction-tuned model, suitable for text generation tasks.

GGUF quantized version of the Gemma 3B model, suitable for local text generation tasks

GGUF quantized version of Mistral-Small-24B-Instruct-2501, suitable for local deployment and text generation tasks.

GGUF-format quantized version of Llama-3.2-3B-Instruct-abliterated, supporting multiple bit quantization options, suitable for text generation tasks.

Yi-Coder-1.5B-Chat-GGUF is the GGUF format model file of 01-ai/Yi-Coder-1.5B-Chat, suitable for text generation tasks.

GGUF format model file for Phi-3.5-mini-instruct, suitable for text generation tasks.

Mistral-Nemo-Instruct-2407-GGUF is the GGUF format quantized version of mistralai/Mistral-Nemo-Instruct-2407, supporting multiple quantization bits (2-bit to 8-bit), suitable for text generation tasks.

GGUF quantized version of Mistral-7B-Instruct-v0.3, suitable for local inference text generation models

A 141-billion parameter Mixture of Experts (MoE) model fine-tuned from Mixtral-8x22B-v0.1, with 35 billion active parameters, primarily designed for English text generation tasks

Saul-Instruct-v1-GGUF is the GGUF format version of the Equall/Saul-Instruct-v1 model, suitable for text generation tasks and supports multiple quantization levels.

GGUF quantized version of the Gemma 2B model, suitable for local deployment and inference

A text ranking model based on Mistral-7B, offering multiple quantized versions for efficient inference.

This is a German/English bilingual model fine-tuned based on Mistral-7B-Instruct-v0.1, quantized in GGUF format with support for multiple quantization levels from 2 to 8 bits.